Discussion of “ Least Angle Regression ” by Efron

Most of this article concerns the uses of LARS and the two related methods in the age-old, " somewhat notorious, " problem of " [a]utomatic model-building algorithms.. . " for linear regression. In the following, I will confine my comments to this notorious problem and to the use of LARS and its relatives to solve it. 1. The implicit assumption. Suppose the response is y, and we collect the m predictors into a vector x, the realized data into an n × m matrix X and the response is the n-vector Y. If P is the projection onto the column space of (1, X), then LARS, like ordinary least squares (OLS), assumes that, for the purposes of model building, Y can be replaced byˆY = P Y without loss of information. In large samples, this is equivalent to the assumption that the conditional distributions F (y|x) can be written as F (y|x) = F (y|x ′ β) (1.1) for some unknown vector β. Efron, Hastie, Johnstone and Tibshirani use this assumption in the definition of the LARS algorithm and in estimating residual variance byˆσ 2 = (I − P)Y 2 /(n − m − 1). For LARS to be reasonable , we need to have some assurance that this particular assumption holds or that it is relatively benign. If this assumption is not benign, then LARS like OLS is unlikely to produce useful results. A more general alternative to (1.1) is F (y|x) = F (y|x ′ B), (1.2) where B is an m × d rank d matrix. The smallest value of d for which (1.2) holds is called the structural dimension of the regression problem [Cook (1998)]. An obvious precursor to fitting linear regression is deciding on the structural dimension, not proceeding as if d = 1. For the diabetes data used